Neuronal Modeling of Cross-Sensory Visual Evoked Magnetoencephalography Responses in the Auditory Cortex.

Previous studies have demonstrated that auditory cortex activity can be influenced by cross-sensory visual inputs. Intracortical laminar recordings in nonhuman primates have suggested a feedforward (FF) type profile for auditory evoked but feedback (FB) type for visual evoked activity in the auditory cortex. To test whether cross-sensory visual evoked activity in the auditory cortex is associated with FB inputs also in humans, we analyzed magnetoencephalography (MEG) responses from eight human subjects (six females) evoked by simple auditory or visual stimuli. In the estimated MEG source waveforms for auditory cortex regions of interest, auditory evoked response showed peaks at 37 and 90 ms and visual evoked response at 125 ms. The inputs to the auditory cortex were modeled through FF- and FB-type connections targeting different cortical layers using the Human Neocortical Neurosolver (HNN), which links cellular- and circuit-level mechanisms to MEG signals. HNN modeling suggested that the experimentally observed auditory response could be explained by an FF input followed by an FB input, whereas the cross-sensory visual response could be adequately explained by just an FB input. Thus, the combined MEG and HNN results support the hypothesis that cross-sensory visual input in the auditory cortex is of FB type. The results also illustrate how the dynamic patterns of the estimated MEG source activity can provide information about the characteristics of the input into a cortical area in terms of the hierarchical organization among areas.

Individualized white matter connectivity of the articulatory pathway: An ultra-high field study.

In current sensorimotor theories pertaining to speech perception, there is a notable emphasis on the involvement of the articulatory-motor system in the processing of speech sounds. Using ultra-high field diffusion-weighted imaging at 7 Tesla, we visualized the white matter tracts connected to areas activated during a simple speech-sound production task in 18 healthy right-handed adults. Regions of interest for white matter tractography were individually determined through 7T functional MRI (fMRI) analyses, based on activations during silent vocalization tasks. These precentral seed regions, activated during the silent production of a lip-vowel sound, demonstrated anatomical connectivity with posterior superior temporal gyrus areas linked to the auditory perception of phonetic sounds. Our study provides a macrostructural foundation for understanding connections in speech production and underscores the central role of the articulatory motor system in speech perception. These findings highlight the value of ultra-high field 7T MR acquisition in unraveling the neural underpinnings of speech.

Characterizing directional dynamics of semantic prediction based on inter-regional temporal generalization.

The event-related potential/field component N400(m) has been widely used as a neural index for semantic prediction. It has long been hypothesized that feedback information from inferior frontal areas plays a critical role in generating the N400. However, due to limitations in causal connectivity estimation, direct testing of this hypothesis has remained difficult. Here, magnetoencephalography (MEG) data was obtained during a classic N400 paradigm where the semantic predictability of a fixed target noun was manipulated in simple German sentences. To estimate causality, we implemented a novel approach based on machine learning and temporal generalization to estimate the effect of inferior frontal gyrus (IFG) on temporal areas. In this method, a support vector machine (SVM) classifier is trained on each time point of the neural activity in IFG to classify less predicted (LP) and highly predicted (HP) nouns and then tested on all time points of superior/middle temporal sub-regions activity (and vice versa, to establish spatio-temporal evidence for or against causality). The decoding accuracy was significantly above chance level when the classifier was trained on IFG activity and tested on future activity in superior and middle temporal gyrus (STG/MTG). The results present new evidence for a model predictive speech comprehension where predictive IFG activity is fed back to shape subsequent activity in STG/MTG, implying a feedback mechanism in N400 generation. In combination with the also observed strong feedforward effect from left STG/MTG to IFG, our findings provide evidence of dynamic feedback and feedforward influences between IFG and temporal areas during N400 generation.

Neuronal modeling of magnetoencephalography responses in auditory cortex to auditory and visual stimuli.

Previous studies have demonstrated that auditory cortex activity can be influenced by crosssensory visual inputs. Intracortical recordings in non-human primates (NHP) have suggested a bottom-up feedforward (FF) type laminar profile for auditory evoked but top-down feedback (FB) type for cross-sensory visual evoked activity in the auditory cortex. To test whether this principle applies also to humans, we analyzed magnetoencephalography (MEG) responses from eight human subjects (six females) evoked by simple auditory or visual stimuli. In the estimated MEG source waveforms for auditory cortex region of interest, auditory evoked responses showed peaks at 37 and 90 ms and cross-sensory visual responses at 125 ms. The inputs to the auditory cortex were then modeled through FF and FB type connections targeting different cortical layers using the Human Neocortical Neurosolver (HNN), which consists of a neocortical circuit model linking the cellular- and circuit-level mechanisms to MEG. The HNN models suggested that the measured auditory response could be explained by an FF input followed by an FB input, and the crosssensory visual response by an FB input. Thus, the combined MEG and HNN results support the hypothesis that cross-sensory visual input in the auditory cortex is of FB type. The results also illustrate how the dynamic patterns of the estimated MEG/EEG source activity can provide information about the characteristics of the input into a cortical area in terms of the hierarchical organization among areas.

Visual Stimuli Modulate Local Field Potentials But Drive No High-Frequency Activity in Human Auditory Cortex.

Neuroimaging studies suggest cross-sensory visual influences in human auditory cortices (ACs). Whether these influences reflect active visual processing in human ACs, which drives neuronal firing and concurrent broadband high-frequency activity (BHFA; >70 Hz), or whether they merely modulate sound processing is still debatable. Here, we presented auditory, visual, and audiovisual stimuli to 16 participants (7 women, 9 men) with stereo-EEG depth electrodes implanted near ACs for presurgical monitoring. Anatomically normalized group analyses were facilitated by inverse modeling of intracranial source currents. Analyses of intracranial event-related potentials (iERPs) suggested cross-sensory responses to visual stimuli in ACs, which lagged the earliest auditory responses by several tens of milliseconds. Visual stimuli also modulated the phase of intrinsic low-frequency oscillations and triggered 15-30 Hz event-related desynchronization in ACs. However, BHFA, a putative correlate of neuronal firing, was not significantly increased in ACs after visual stimuli, not even when they coincided with auditory stimuli. Intracranial recordings demonstrate cross-sensory modulations, but no indication of active visual processing in human ACs.

Role of articulatory motor networks in perceptual categorization of speech signals: a 7T fMRI study.

Speech and language processing involve complex interactions between cortical areas necessary for articulatory movements and auditory perception and a range of areas through which these are connected and interact. Despite their fundamental importance, the precise mechanisms underlying these processes are not fully elucidated. We measured BOLD signals from normal hearing participants using high-field 7 Tesla fMRI with 1-mm isotropic voxel resolution. The subjects performed 2 speech perception tasks (discrimination and classification) and a speech production task during the scan. By employing univariate and multivariate pattern analyses, we identified the neural signatures associated with speech production and perception. The left precentral, premotor, and inferior frontal cortex regions showed significant activations that correlated with phoneme category variability during perceptual discrimination tasks. In addition, the perceived sound categories could be decoded from signals in a region of interest defined based on activation related to production task. The results support the hypothesis that articulatory motor networks in the left hemisphere, typically associated with speech production, may also play a critical role in the perceptual categorization of syllables. The study provides valuable insights into the intricate neural mechanisms that underlie speech processing.

Comparing auditory and visual aspects of multisensory working memory using bimodally matched feature patterns.

Working memory (WM) reflects the transient maintenance of information in the absence of external input, which can be attained via multiple senses separately or simultaneously. Pertaining to WM, the prevailing literature suggests the dominance of vision over other sensory systems. However, this imbalance may be stemming from challenges in finding comparable stimuli across modalities. Here, we addressed this problem by using a balanced multisensory retro-cue WM design, which employed combinations of auditory (ripple sounds) and visuospatial (Gabor patches) patterns, adjusted relative to each participant's discrimination ability. In three separate experiments, the participant was asked to determine whether the (retro-cued) auditory and/or visual items maintained in WM matched or mismatched the subsequent probe stimulus. In Experiment 1, all stimuli were audiovisual, and the probes were either fully mismatching, only partially mismatching, or fully matching the memorized item. Experiment 2 was otherwise same as Experiment 1, but the probes were unimodal. In Experiment 3, the participant was cued to maintain only the auditory or visual aspect of an audiovisual item pair. In two of the three experiments, the participant matching performance was significantly more accurate for the auditory than visual attributes of probes. When the perceptual and task demands are bimodally equated, auditory attributes can be matched to multisensory items in WM at least as accurately as, if not more precisely than, their visual counterparts.

Role of Articulatory Motor Networks in Perceptual Categorization of Speech Signals: A 7 T fMRI Study.

BACKGROUND: The association between brain regions involved in speech production and those that play a role in speech perception is not yet fully understood. We compared speech production related brain activity with activations resulting from perceptual categorization of syllables using high field 7 Tesla functional magnetic resonance imaging (fMRI) at 1-mm isotropic voxel resolution, enabling high localization accuracy compared to previous studies. METHODS: Blood oxygenation level dependent (BOLD) signals were obtained in 20 normal hearing subjects using a simultaneous multi-slice (SMS) 7T echo-planar imaging (EPI) acquisition with whole-head coverage and 1 mm isotropic resolution. In a speech production localizer task, subjects were asked to produce a silent lip-round vowel /u/ in response to the visual cue "U" or purse their lips when they saw the cue "P". In a phoneme discrimination task, subjects were presented with pairs of syllables, which were equiprobably identical or different along an 8-step continuum between the prototypic /ba/ and /da/ sounds. After the presentation of each stimulus pair, the subjects were asked to indicate whether the two syllables they heard were identical or different by pressing one of two buttons. In a phoneme classification task, the subjects heard only one syllable and asked to indicate whether it was /ba/ or /da/. RESULTS: Univariate fMRI analyses using a parametric modulation approach suggested that left motor, premotor, and frontal cortex BOLD activations correlate with phoneme category variability in the /ba/-/da/ discrimination task. In contrast, the variability related to acoustic features of the phonemes were the highest in the right primary auditory cortex. Our multivariate pattern analysis (MVPA) suggested that left precentral/inferior frontal cortex areas, which were associated with speech production according to the localizer task, play a role also in perceptual categorization of the syllables. CONCLUSIONS: The results support the hypothesis that articulatory motor networks in the left hemisphere that are activated during speech production could also have a role in perceptual categorization of syllables. Importantly, high voxel-resolution combined with advanced coil technology allowed us to pinpoint the exact brain regions involved in both perception and production tasks.

Spectrotemporal content of human auditory working memory represented in functional connectivity patterns.

Recent research suggests that working memory (WM), the mental sketchpad underlying thinking and communication, is maintained by multiple regions throughout the brain. Whether parts of a stable WM representation could be distributed across these brain regions is, however, an open question. We addressed this question by examining the content-specificity of connectivity-pattern matrices between subparts of cortical regions-of-interest (ROI). These connectivity patterns were calculated from functional MRI obtained during a ripple-sound auditory WM task. Statistical significance was assessed by comparing the decoding results to a null distribution derived from a permutation test considering all comparable two- to four-ROI connectivity patterns. Maintained WM items could be decoded from connectivity patterns across ROIs in frontal, parietal, and superior temporal cortices. All functional connectivity patterns that were specific to maintained sound content extended from early auditory to frontoparietal cortices. Our results demonstrate that WM maintenance is supported by content-specific patterns of functional connectivity across different levels of cortical hierarchy.

Cortical depth profiles of auditory and visual 7 T functional MRI responses in human superior temporal areas.

Invasive neurophysiological studies in nonhuman primates have shown different laminar activation profiles to auditory vs. visual stimuli in auditory cortices and adjacent polymodal areas. Means to examine the underlying feedforward vs. feedback type influences noninvasively have been limited in humans. Here, using 1-mm isotropic resolution 3D echo-planar imaging at 7 T, we studied the intracortical depth profiles of functional magnetic resonance imaging (fMRI) blood oxygenation level dependent (BOLD) signals to brief auditory (noise bursts) and visual (checkerboard) stimuli. BOLD percent-signal-changes were estimated at 11 equally spaced intracortical depths, within regions-of-interest encompassing auditory (Heschl's gyrus, Heschl's sulcus, planum temporale, and posterior superior temporal gyrus) and polymodal (middle and posterior superior temporal sulcus) areas. Effects of differing BOLD signal strengths for auditory and visual stimuli were controlled via normalization and statistical modeling. The BOLD depth profile shapes, modeled with quadratic regression, were significantly different for auditory vs. visual stimuli in auditory cortices, but not in polymodal areas. The different depth profiles could reflect sensory-specific feedforward versus cross-sensory feedback influences, previously shown in laminar recordings in nonhuman primates. The results suggest that intracortical BOLD profiles can help distinguish between feedforward and feedback type influences in the human brain. Further experimental studies are still needed to clarify how underlying signal strength influences BOLD depth profiles under different stimulus conditions.

Multilayer Network Analysis across Cortical Depths in Resting-State 7T fMRI.

In graph theory, "multilayer networks" represent systems involving several interconnected topological levels. A neuroscience example is the hierarchy of connections between different cortical depths or "lamina". This hierarchy is becoming non-invasively accessible in humans using ultra-high-resolution functional MRI (fMRI). Here, we applied multilayer graph theory to examine functional connectivity across different cortical depths in humans, using 7T fMRI (1-mm3 voxels; 30 participants). Blood oxygenation level dependent (BOLD) signals were derived from five depths between the white matter and pial surface. We then compared networks where the inter-regional connections were limited to a single cortical depth only ("layer-by-layer matrices") to those considering all possible connections between regions and cortical depths ("multilayer matrix"). We utilized global and local graph theory features that quantitatively characterize network attributes such as network composition, nodal centrality, path-based measures, and hub segregation. Detecting functional differences between cortical depths was improved using multilayer connectomics compared to the layer-by-layer versions. Superficial aspects of the cortex dominated information transfer and deeper aspects clustering. These differences were largest in frontotemporal and limbic brain regions. fMRI functional connectivity across different cortical depths may contain neurophysiologically relevant information. Multilayer connectomics could provide a methodological framework for studies on how information flows across this hierarchy.

Do sparse brain activity patterns underlie human cognition?

Accumulating multivariate pattern analysis (MVPA) results from fMRI studies suggest that information is represented in fingerprint patterns of activations and deactivations during perception, emotions, and cognition. We postulate that these fingerprint patterns might reflect neuronal-population level sparse code documented in two-photon calcium imaging studies in animal models, i.e., information represented in specific and reproducible ensembles of a few percent of active neurons amidst widespread inhibition in neural populations. We suggest that such representations constitute a fundamental organizational principle via interacting across multiple levels of brain hierarchy, thus giving rise to perception, emotions, and cognition.

Association of lifetime major depressive disorder with enhanced attentional sensitivity measured with P3 response in young adult twins.

Major depression is associated with alterations in the auditory P3 event-related potential (ERP). However, the persistence of these abnormalities after recovery from depressive episodes, especially in young adults, is not well known. Furthermore, the potential influence of substance use on this association is poorly understood. Young adult twin pairs (N = 177) from the longitudinal FinnTwin16 study were studied with a psychiatric interview, and P3a and P3b ERPs elicited by task-irrelevant novel sounds and targets, respectively. Dyadic linear mixed-effect models were used to distinguish the effects of lifetime major depressive disorder from familial factors and effects of alcohol problem drinking and tobacco smoking. P3a amplitude was significantly increased and P3b latency decreased, in individuals with a history of lifetime major depression, when controlling the fixed effects of alcohol abuse, tobacco, gender, twins' birth order, and zygosity. These results suggest that past lifetime major depressive disorder may be associated with enhanced attentional sensitivity.

Weighted Blind Source Separation Can Decompose the Frequency Mismatch Response by Deviant Concatenation: An MEG Study.

The mismatch response (MMR) is thought to be a neurophysiological measure of novel auditory detection that could serve as a translational biomarker of various neurological diseases. When recorded with electroencephalography (EEG) or magnetoencephalography (MEG), the MMR is traditionally extracted by subtracting the event-related potential/field (ERP/ERF) elicited in response to "deviant" sounds that occur randomly within a train of repetitive "standard" sounds. However, there are several problems with such a subtraction, which include increased noise and the neural adaptation problem. On the basis of the original theory underlying MMR (i.e., the memory-comparison process), the MMR should be present only in deviant epochs. Therefore, we proposed a novel method called weighted-BSS T/k, which uses only the deviant response to derive the MMR. Deviant concatenation and weight assignment are the primary procedures of weighted-BSS T/k, which maximize the benefits of time-delayed correlation. We hypothesized that this novel weighted-BSS T/k method highlights responses related to the detection of the deviant stimulus and is more sensitive than independent component analysis (ICA). To test this hypothesis and the validity and efficacy of the weighted-BSS T/k in comparison with ICA (infomax), we evaluated the methods in 12 healthy adults. Auditory stimuli were presented at a constant rate of 2 Hz. Frequency MMRs at a sensor level were obtained from the bilateral temporal lobes with the subtraction approach at 96-276 ms (the MMR time range), defined based on spatio-temporal cluster permutation analysis. In the application of the weighted-BSS T/k, the deviant responses were given a constant weight using a rectangular window on the MMR time range. The ERF elicited by the weighted deviant responses demonstrated one or a few dominant components representing the MMR that fitted well with that of the sensor space analysis using the conventional subtraction approach. In contrast, infomax or weighted-infomax revealed many minor or pseudo components as constituents of the MMR. Our single-trial, contrast-free approach may assist in using the MMR in basic and clinical research, and it opens a new and potentially useful way to analyze event-related MEG/EEG data.

Behavioral Experience-Sampling Methods in Neuroimaging Studies With Movie and Narrative Stimuli.

Movies and narratives are increasingly utilized as stimuli in functional magnetic resonance imaging (fMRI), magnetoencephalography (MEG), and electroencephalography (EEG) studies. Emotional reactions of subjects, what they pay attention to, what they memorize, and their cognitive interpretations are all examples of inner experiences that can differ between subjects during watching of movies and listening to narratives inside the scanner. Here, we review literature indicating that behavioral measures of inner experiences play an integral role in this new research paradigm via guiding neuroimaging analysis. We review behavioral methods that have been developed to sample inner experiences during watching of movies and listening to narratives. We also review approaches that allow for joint analyses of the behaviorally sampled inner experiences and neuroimaging data. We suggest that building neurophenomenological frameworks holds potential for solving the interrelationships between inner experiences and their neural underpinnings. Finally, we tentatively suggest that recent developments in machine learning approaches may pave way for inferring different classes of inner experiences directly from the neuroimaging data, thus potentially complementing the behavioral self-reports.

A novel time-delayed correlation method decomposes mismatch response without using subtraction.

The mismatch response (MMR) is thought to be a neurophysiological measure of novel auditory detection that could serve as a translational biomarker of various neurological diseases. When recorded with electroencephalography (EEG) or magnetoencephalography (MEG), the MMR is traditionally extracted by subtracting the event-related potential/field (ERP/ERF) elicited in response to "deviant" sounds that occur randomly within a train of repetitive "standard" sounds. To overcome the limitations of this subtraction procedure, we propose a novel method which we call weighted-BSST/k, which uses only the deviant response to derive the MMR. We hypothesized that this novel weighted-BSST/k method highlights responses related to the detection of the deviant stimulus and is more sensitive than independent component analysis (ICA). To test this hypothesis and the validity and efficacy of the weighted-BSST/k in comparison with ICA (infomax), we evaluated the methods in 12 healthy adults. Auditory stimuli were presented at a constant rate of 2 Hz. Frequency MMRs at a sensor level were obtained from the bilateral temporal lobes with the subtraction approach at 96-276 ms (the MMR time range), defined on the basis of spatio-temporal cluster permutation analysis. In the application of the weighted-BSST/k, the deviant responses were given a constant weight on the MMR time range. The ERF elicited by the weighted deviant responses demonstrated one or a few dominant components representing the MMR with a high signal-to-noise ratio and similar topography to that of the sensor space analysis using the subtraction approach. In contrast, infomax or weighted-infomax revealed many minor or pseudo components as constituents of the MMR. Our new approach may assist in using the MMR in basic and clinical research.Clinical Relevance-Our proposed method opens a new and potentially useful way to analyze event-related MEG/EEG data.

Synchronization patterns reveal neuronal coding of working memory content.

Neuronal oscillations are suggested to play an important role in auditory working memory (WM), but their contribution to content-specific representations has remained unclear. Here, we measure magnetoencephalography during a retro-cueing task with parametric ripple-sound stimuli, which are spectrotemporally similar to speech but resist non-auditory memory strategies. Using machine learning analyses, with rigorous between-subject cross-validation and non-parametric permutation testing, we show that memorized sound content is strongly represented in phase-synchronization patterns between subregions of auditory and frontoparietal cortices. These phase-synchronization patterns predict the memorized sound content steadily across the studied maintenance period. In addition to connectivity-based representations, there are indices of more local, "activity silent" representations in auditory cortices, where the decoding accuracy of WM content significantly increases after task-irrelevant "impulse stimuli." Our results demonstrate that synchronization patterns across auditory sensory and association areas orchestrate neuronal coding of auditory WM content. This connectivity-based coding scheme could also extend beyond the auditory domain.

Boundary element fast multipole method for modeling electrical brain stimulation with voltage and current electrodes.

Objective. To formulate, validate, and apply an alternative to the finite element method (FEM) high-resolution modeling technique for electrical brain stimulation-the boundary element fast multipole method (BEM-FMM). To include practical electrode models for both surface and embedded electrodes.Approach. Integral equations of the boundary element method in terms of surface charge density are combined with a general-purpose fast multipole method and are expanded for voltage, shunt, current, and floating electrodes. The solution of coupled and properly weighted/preconditioned integral equations is accompanied by enforcing global conservation laws: charge conservation law and Kirchhoff's current law.Main results.A sub-percent accuracy is reported as compared to the analytical solutions and simple validation geometries. Comparison to FEM considering realistic head models resulted in relative differences of the electric field magnitude in the range of 3%-6% or less. Quantities that contain higher order spatial derivatives, such as the activating function, are determined with a higher accuracy and a faster speed as compared to the FEM. The method can be easily combined with existing head modeling pipelines such as headreco or mri2mesh.Significance.The BEM-FMM does not rely on a volumetric mesh and is therefore particularly suitable for modeling some mesoscale problems with submillimeter (and possibly finer) resolution with high accuracy at moderate computational cost. Utilizing Helmholtz reciprocity principle makes it possible to expand the method to a solution of EEG forward problems with a very large number of cortical dipoles.

Auditory cues facilitate object movement processing in human extrastriate visual cortex during simulated self-motion: A pilot study.

Visual segregation of moving objects is a considerable computational challenge when the observer moves through space. Recent psychophysical studies suggest that directionally congruent, moving auditory cues can substantially improve parsing object motion in such settings, but the exact brain mechanisms and visual processing stages that mediate these effects are still incompletely known. Here, we utilized multivariate pattern analyses (MVPA) of MRI-informed magnetoencephalography (MEG) source estimates to examine how crossmodal auditory cues facilitate motion detection during the observer's self-motion. During MEG recordings, participants identified a target object that moved either forward or backward within a visual scene that included nine identically textured objects simulating forward observer translation. Auditory motion cues 1) improved the behavioral accuracy of target localization, 2) significantly modulated the MEG source activity in the areas V2 and human middle temporal complex (hMT+), and 3) increased the accuracy at which the target movement direction could be decoded from hMT+ activity using MVPA. The increase of decoding accuracy by auditory cues in hMT+ was significant also when superior temporal activations in or near auditory cortices were regressed out from the hMT+ source activity to control for source estimation biases caused by point spread. Taken together, these results suggest that parsing object motion from self-motion-induced optic flow in the human extrastriate visual cortex can be facilitated by crossmodal influences from auditory system.